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Abstract: 

Picture  someone  walking  from  left  to  right.  During  one  step  (intra-step)  we  treat  them  as 
a  simple  pendulum.  This  model  is  called  the  rimless  wheel  in  the  literature.  We  analyze 
this  model  intra-step  using  dynamic  programming  to  find  the  optimum  energy  profile 
based  on  time  and  energy  cost.  We  then  analyze  the  problem  inter-step  for  the  ideal 
stepsize  based  on  time  cost  alone,  i.e.,  without  foot  collision  (energy)  losses. 
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Figure  1:  Phase  Plane  of  the  Inverted  Pendulum 


Introduction:  Unpowered  walking 

We  derive  the  pendulum  equation  as  shown  in  Appendix  A  (see  Figure  9  for  coordinate 
system).  From  equation  (17),  divide  through  by  m  and  set  g  =  /  =  1  and  the  scalar  energy 
lines  of  the  pendulum  as  plotted  in  Figure  1  are  given  by: 


(1) 


A  valid  trajectory  for  our  walker  starts  on  the  right-hand  side  of  Figure  1  where  6  >  /Tand 
travels  right  to  left  (as  opposed  to  the  motion  of  the  pendulum  itself,  which  is  clockwise 
from  about  10  to  2  o’clock).  If  there  is  enough  energy,  the  pendulum  rotates  over  the  top 

7Z 

until  foot  collision  at  some  angle,  6  -  a  ,  before  it  hits  the  ground  at  6  =  — .  Energy 

levels  (which  are  directly  related  to  the  relative  speed  of  the  pendulum  at  a  given  angle) 
are  depicted  as  contour  lines  on  the  above  map.  We  ignore  the  angles  where  the 
pendulum  is  below  the  ground,  because  obviously,  that  doesn’t  mean  a  whole  lot  to  us. 
The  red  line  is  the  separatrix  at  E=l.  Above  the  red  line,  the  pendulum  doesn’t  have 
enough  energy  to  rotate  over  the  top  and  the  behavior  fundamentally  changes  to  an 
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oscillation  about  the  other  fixed  point  at  9  —  0  (not  shown).  For  us,  this  simply  means 
the  pendulum  falls  backwards  until  it  hits  the  ground. 

Powered  Walking 

Now,  we’d  like  to  create  an  algorithm  for  calculating  the  minimum  cost  in  time  and 
energy  to  operate  a  powered  pendulum  given  by: 

9  +  sin  9  -  u  (2) 

Powering  the  pendulum  adds  energy  to  the  system;  hence  it  is  no  longer  a  conservative 
system  (see  Strogatz,  p.  170-173).  However,  we  can  calculate  the  change  in  energy  by 
taking  the  time  derivative  of  the  conservative  equation  (1)  and  comparing  the  result  to  the 
powered  system  in  (2): 

—  =  —  —  02-cos0  =  0(0  +  sin0)  =  u0 
dt  dt\2  J  v  ' 

We  see  from  the  above  equation  that  at  lower  speeds,  9 ,  a  higher  torque,  u,  is  required  for 
a  given  energy  change,  A E .  For  convenience,  we  will  treat  the  powered  system  as  being 
intennittently  powered  but  then  coasting  along  the  constant  energy  contours  shown.  This 
gives  us  a  starting  point  for  calculating  the  cost  of  energy  change.  We  use  the  simple 
case  where  the  pendulum  will  have  enough  energy  to  make  it  over  the  hump  at  9  —  n . 
(This  avoids  nasty  division  by  zero  problems).  Furthermore,  we  will  calculate  cost  based 
upon  A E ,  not  on  u.  This  will  free  us  from  having  to  adjust  our  answers  depending  on  9 . 


Figure  2:  Phase  plane  of  the  pendulum  walker  with  dynamic  programming 

Dynamic  Programming 

The  algorithm  we’d  like  to  develop  is  based  upon  a  dynamic  programming  example 
shown  in  Appendix  B.  Let’s  take  what  we’ve  learned  there  and  apply  it  to  the  pendulum 
walker.  Instead  of  4  possible  choices  per  node,  we  designed  the  problem  to  only  have 
two  choices  per  node:  1)  Stay  on  the  energy  level  you  are  currently  at— dotted  line  or  2) 
Move  up  to  the  next  level— solid  line.  Starting  from  the  upper  right  in  Figure  2  and 
working  your  way  left,  you  can  see  how  this  would  look.  We  do  our  accounting  with  the 
functional  equation: 

f]  =  min  j/(/  +  f. |  for  i  ^  end 


(3) 


This  all  comes  down  to  how  you  number  the  nodes  and  build  your  Cost  matrix.  As  you 
can  see  in  Figure  2,  3,  and  4  there  are  five  Energy  levels:  El...Es  and  n  increments  of  6 . 

Starting  from  the  upper  right  (Node:  1  in  Figure  3)  which  is  at  energy  level:  Ex  and 
angle:  6X  (as  shown  in  Figure  3)  and  moving  left  you  have  two  choices  as  depicted  by  the 

two  arrows.  You  can  head  to  node  six  by  staying  on  the  same  E-  level,  which  would  be 
node  number  m+5  in  general,  where  m  is  the  current  node  number.  Or  you  may  move  to 
node  number  seven  (m+ 6  in  general)  by  moving  “up”  to  Energy  level  .  If  you  stay  on 
the  same  energy  contour  then  your  only  cost  is  the  time  cost  At  to  go  from  6n  to  8n+] . 

Now  keep  in  mind  that  the  angle  Ad  is  a  constant,  but  the  time  cost,  At ,  varies  depending 
on  where  you  are  at  on  the  graph.  If  you  move  up  an  energy  level,  then  you  incur  a  cost 
equal  to  both  At  and  A E  . 

At 

Cost  =  depending  on  path  taken  (4) 

At +  AE 

this  corresponds  to  an  arc  length  as  defined  in  Appendix  B: 

t(i,j)=Cost(m,m+n+4)  (5) 

and  the  total  cost  for  n= 1:2  is 

Total _Cost(m,n)=Cost(m,n)+f(m+n+4);  (6) 


Figure  3:  Node  Numbering  Scheme 


We  then  proceed  as  shown  in  the  appendix,  and  when  we  are  done  we  create  a  decision 
tree  that  shows  what  the  optimal  path  should  be  for  different  starting  positions  in  phase 
space.  Generally  we  will  start  at  the  upper  right  hand  side  of  the  graph  and  move  right  to 
left.  Let’s  say,  for  example,  that  you  started  at  the  lowest  energy  level,  £=1.01  (shown  as 
the  top  curve  of  Figure  4).  This  energy  level  is  barely  beyond  the  separatrix  at  E=l,  and 
this  means  that  you  will  make  it  over  the  top,  albeit  very  slowly.  The  graph  shows  that 


you  should  move  “up”  one  level  (move  down  on  the  graph)  to  £=1.126.  Basically,  the 
graph  is  telling  us  what  should  be  intuitively  obvious:  at  low  energy  levels,  it  will  take  so 
long  to  get  “over  the  hump”  to  the  other  side,  you  might  as  well  spend  the  energy  and 
gain  the  time  savings  of  being  on  a  higher  energy  level  (read:  higher  speed  at  a  given 
angle)  and  furthermore,  you  should  do  this  as  quickly  as  possible.  What  might  not  be  as 
obvious  is  that  this  logic  has  limits:  there  comes  a  point  at  which  it  is  no  longer  worth  it 
to  expend  the  energy  to  reach  a  new  £-level,  i.e.,  higher,  speed.  Look  at  the  £=1.01 
energy  curve  as  we  move  over  the  hump  to  the  left.  There  comes  a  point  (around 
theta=2.8)  where  it  is  advisable  to  stay  on  this  same  energy  curve.  You  could  think  of 
this  with  the  logic  that  if  you’re  already  over  the  hump,  what’s  the  use  in  spending  the 
additional  energy  now?  You’ve  already  come  so  far! 


Figure  4:  Dynamic  Programming  Results 


Optimum  Stepwidth  without  Collision  Loss  or  Best  Angle 

Let’s  assume,  for  a  moment,  that  you  could  create  a  walking  vehicle  that  had  no  collision 
loss.  We  could  argue  all  day  until  we’re  blue  in  the  face  about  what  device  could 
possibly  do  this,  but  the  bottom  line  is  this:  what  would  be  the  fastest  (average  forward 
velocity)  step  size  if  we  could?  This  provides  a  rock-bottom  energy  analysis  that  should 
prove  useful  as  a  guide  for  building  practical  walking  systems. 


Figure  5:  Pendulum  Walker 


To  solve  the  problem,  we  must  find  the  average  velocity,  v  ,  but  first  we  need  to  find  v. 
Looking  at  Figure  5,  we  see  the  familiar  transformation  from  Cartesian  to  cylindrical 


2 

coordinates  .  The  velocity,  v,  we  are  interested  is  along  the  y-direction  and  is  given  by 
v  =  r6 cos  9  so  if  r=\,  then 

v-9cos9  (7) 

We  know  from  past  work  that  (recall  that  9  is  negative): 

9  =  -y/2-jE  +  cos&  (8) 

Substituting  (8)  into  (7),  we  obtain: 

v  =  E  +  co$6  cos 6*  (9) 

We  can  see  a  graph  of  this  function  in  Figure  6.  Basically,  we  get  a  cosine  penalty  as  we 
3  jz  jz 

approach  —  from  the  left  or  —  from  the  right.  This  corresponds  to  the  situation  in 

Figure  5  where  we  are  going  to  hit  the  ground  to  the  left  or  to  the  right.  At  those  times 
you  are  moving  in  an  up  and  down  direction  and  not  left  to  right  (it’s  like  when  you’re 
doing  the  splits),  so  it’s  clear  that  you  would  like  to  be  centered  around  the  vertical: 

9  —  n ,  where  the  cosine  penalty  is  minimal  (most  of  your  rotational  motion  translates  to 
forward  motion  here).  What’s  not  clear  from  the  pictures  is  why  the  linear  velocity,  v, 
has  a  minimum  at  9  -n .  The  reason  for  that  is  9  has  a  minimum  at  9  -  n .  Why?  Well, 
you  could  look  at  a  graph  of  equation  (8),  or  you  could  use  common  sense  to  think  that 
gravity  is  working  against  the  pendulum  as  it  approaches  its  peak  and  its  Potential  Energy 
(which  is  proportional  to  height:  cos (9)  )  is  maximum  there,  and  since  the  pendulum  is  a 

conservative  system,  its  Kinetic  Energy  (which  is  proportional  to  92 )  must  be  minimum 
there  because  the  overall  energy  must  remain  constant.  This  creates  an  interesting 
scenario:  at  low  energies  we  get  a  situation  like  you  see  in  Figure  6.  This  is  a  graph  of  v 
with  E=  1.01,  so  the  pendulum  is  barely  above  the  separatrix.  This  means  that  it  slows 
down  a  whole  lot  near  the  peak. 


E=1 .25 


Figure  6,  Linear  velocity,  v,  as  a  function  of  9  (in  blue)  plotted  with  cosine  (in  green) 

With  a  situation  like  we  see  in  Figure  6,  we  might  as  well  take  a  step,  because  if  we  are 
not  losing  anything  at  collision  it  makes  sense  to  increase  our  average  speed  by  taking  in 
the  dual  peaks  of  the  velocity  curve,  v,  to  the  left  and  right  of  9  —  n .  At  higher  values  of 
E  as  shown  in  Figure  6(b),  we  start  to  see  the  minimum  at  9  =  n  become  less  pronounced 
until  finally,  in  Figure  7  at  £=1.5  the  minimum  has  become  a  maximum. 


Never  mind  that  the  actual  position  of  the  walker  is  in  the  upper  half  plane:  the  math  will  take  care  of  that. 


E=1 .5 


Figure  7:  Linear  velocity,  v,  as  a  function  of  6  at  £=1.5 

Now,  we  shouldn’t  confuse  anyone,  |#|  always  has  a  minimum  at  9  =  n ,  but  we’re 

plotting,  v,  which  has  the  cosine  penalty  tacked  on  multiplicatively.  What’s  happening  is 
that  at  higher  and  higher  energies,  the  change  in  potential  energy  due  to  the  pendulum 
swinging  around  is  relatively  small  compared  to  the  total  energy.  This  means  that  E, 
which  is  made  up  mostly  of  kinetic  energy  now,  is  not  affected  much  by  changes  in  9 , 
which  means  9  is  then  a  relatively  flat  (approximately  constant)  function  with  respect  to 
6 .  This  causes  v  «  -  cost? ,  as  plotted  in  Figure  7,  for  9  ~  n  and  is  =  1.5  in  equation  (9). 
Thus  the  minimum  at  0  —  n  has  disappeared  entirely,  indeed  it  is  now  a  maximum  and 
there  is  no  point  in  taking  a  step  further,  you  might  as  well  run  or  roll. 

Heuristically  now,  you  should  have  a  sense  of  what  happens.  We  are  left  with  a  couple 
questions:  First,  how  do  we  find  out  what  is  the  best  step  size  for  a  given  energy  level, 

El  Secondly,  how  do  we  calculate  the  maximum  energy  (seen  in  the  graphs  as  occurring 
at  E=  1.5)  where  it  makes  sense  to  walk? 

Optimal  Step  Size 

The  first  question  is  best  answered  by  taking  a  look  at  how  we  calculate  averages: 

Recall: 

v,=7(v.+v,+...v,)  (10) 

n 

where  n  is  the  number  of  samples  taken.  How  would  you  know  if  taking  next  bigger 
stepsize  is  better?  We  assume  that  the  stepsize  is  centered  around  9  —  n ,  and  we  iterate 
so  that  the  stepsize  is  incrementally  bigger.  Thus  if 

vn+1>vn  (11) 

we  know  that  a  bigger  stepsize  has  a  better  average  linear  velocity.  From  Figure  2,  we 
see  that  there  will  come  a  point  where  it  is  no  longer  better  to  take  a  bigger  step.  From 
(10)  we  see  that 
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v„+ 1  =■ 


n  + 


n+ 1 


1  ^  1 

—  >  v.  + - V 

IT  n  +  l 


H+l 


1  _  1 

-V„  H - V 


«  + 1  ”  n  + 1 


W+l 


so  if  we  substitute  vn+1  from  above  into  (1 1)  we  get: 


1 


-V~+ - V„+l  >Vn 


n  + 1  n  + 1 


and  re-arranging  terms,  we  find 


n  -(«  +  !)_  1 


v.  >  -  - 


n  + 1  72  +  1 

finally,  the  condition  for  satisfying  (1 1)  is 


v  n+ 1 


V„+l  >  Vn 


(12) 


In  other  words,  if  the  next  incremental  velocity  is  greater  than  your  average,  take  it. 


This  can  be  seen  graphically  by  looking  at  the  average  as  we  increase  the  stepsize. 
Figure  8  contains  the  right  half  of  the  graph  of  linear  velocity,  v,  as  seen  in  Figure  6 
through  Figure  7.  Since  those  graphs  are  symmetric,  we  can  take  the  average  over  the 
right  half  side,  and  it  will  be  the  same  as  the  average  over  the  whole  curve.  So  we  see 
that  the  average  velocity  increases  until  it  reaches  the  condition  in  equation  (12)  where 
the  curves  cross,  and  from  then  on  the  average  velocity  decreases. 


E=1 .01 


theta 


Figure  8:  Optimum  Stepwidth  at  E=  1.01  is  about  138° 


The  optimum  stepwidth  is  calculated  by  taking  the  angle  between  where  the  curves  cross, 
9cross ,  and  n  and  multiplying  that  quantity  by  two  (by  symmetry  with  the  left  hand  side 
of  the  curve). 


«L,=2(0™-*-)  (13) 

The  same  trend  we  saw  in  Figures  6  through  7  continues  here,  at  higher  energy  levels,  the 
optimum  stepwidth  gets  narrower  and  narrower,  until  at  E=  1 .5,  it  makes  no  sense  to  take 
a  step  at  all  (stepwidth  approaches  zero). 


Maximum  Walking  Energy 

We’ve  seen  clues  graphically  that  E=1.5  is  the  maximum  energy  where  it  makes  sense  to 
walk.  Is  there  another  way  we  can  find  this  value?  We  seek  an  answer  to  the  second 
question  we  posed  above.  We  start  from  the  equation  for  the  forward  velocity,  equation 
(9),  which  we  repeat  here: 

v  =  -y[2\jE  +  cos#  cos # 

If  we  examine  the  graphs  in  Figures  6  and  7  above,  we  see  that  the  minimum  at 
9-71  becomes  a  maximum  as  we  change  energy.  If  we  take  the  derivative  of  v  with 
respect  to  6 ,  we  see  that 

dv  V2  _  sin# 

—  =  — cos#— .= 
d6  2  Vfs+cos# 


->/2  sin  o4e  +  cos  # 


Simplifying: 


dv  V2  .  n(2E  +  3cos9^ 

—  = - sin#  .  =- 

dO  2  ^  +  cos  # 


(14) 


You  can  see  that  at  #  =  n  there  is  always  extremum,  because  sin  n  =  0 .  We  can  now 

examine  the  sign  of  —  to  test  if  v  has  a  minimum  or  a  maximum  at  #  =  n .  Recall  that 
dd 

if  changes  from  +  to  -  at  0  —  n,  then  v  has  a  local  maximum  there.  Recall  that  sin  # 
dd 

changes  from  +  to  -  through#  =  n .  So  what  happens  there  depends  on  the  value  of  E  in 
the  numerator  and  denominator  of  the  term  in  brackets.  Clearly,  E  >  1  so  that  nothing 
strange  happens  under  the  square  root.  Now  we  are  left  with  the  numerator:  If  the 

numerator,  2 E  +  cos  # ,  is  positive  then  the  sign  of  —  changes  from  +  to  -  and  we  have  a 

dO 

local  maximum  there.  The  condition  for  this  local  maximum  is: 

2E +  3  cos  n  >  0 

simplifying: 

E>-  (15) 

2 

which  is  just  what  we  saw  graphically:  above  E=  1.5,  it  makes  no  sense  to  take  a  step. 


We  can  check  this  result  with  the  second  derivative  test: 


d2v  V2 
— r-  =  — cos# 
dd2  2 


2fs +  3cos# 
VeT cos# 


a 

H - sin# 

2 


f  \ 

-3  sin  #  1  .  (2E  +  cos#) 

,  +  — sin#- - f 

y/E  +  cos  9  2  (E  +  cos9)2y 


(16) 


We  can  see  immediately  that  the  whole  second  tenn  goes  to  zero  at  0  —  n  because 
sin  n  —  0 .  This  means  that  the  inflection  point  is  again  where  the  numerator  of  the  first 
term  equals  zero,  i.e.,  2E  +  cos0-O.  This  is,  of  course,  at  E=  1.5. 

Appendix  A:  The  Simple  Pendulum 

Cough,  cough,  do  you  smell  the  dust?  This  is  a  hoary  problem,  but  it  is  the  basic 
dynamics  of  our  walker.  Let’s  start  by  deriving  the  equation  of  motion  using  the 

•5 

conservation  of  energy.  We  assume  a  rigid,  massless  rod. 


First  of  all  the  speed  of  the  the  pendulum  bob  is  given  by: 

v  =  l0 

The  kinetic  energy  of  the  bob  is  then: 

T  =  —mv2  =  -ml20 2 
2  2 

We  have  the  potential  energy  also: 

V  =  mgh  =  mg  (I  - 1  cos  0) 

where  we  have  chosen  the  datum,  where  the  potential  energy  is  zero,  to  be  at  the  bottom 
of  the  pendulum’s  swing.  This  is  an  arbitrary  choice.  We  can  just  as  easily  choose  the 
datum  to  be  at  the  ground  level.  Then  V  =  -mgl  cos  0  The  total  energy  is  then  given  by: 


E 


T  +  V  =  —ml202 
2 


-  mgl  cos  0 


(17) 


dE 

Either  way,  the  conservation  of  energy  says  that  —  =  0 ,  and  we  obtain: 

dt 

ml2 00  +  mgl0  sin  0  =  0 

where  we  used  the  chain  rule.  This  has  two  solutions,  either  0-0  always,  or 


0  +  —sin0  =  0 
l 

which  is  the  familiar  equation  of  motion. 


Appendix  B:  Dynamic  Programming* 4 


1  This  derivation  can  be  found  online  at  http://www.scar.utoronto.ca/~pat/fun/NEWT3D/PDF/ 

4  This  example  is  from  Dynamic  Programming:  Models  and  Applications  Eric  V.  Denardo 
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Figure  10:  A  Directed  Network 


This  directed  network  example  consists  of  a  set  of  nine  nodes  and  a  set  of  1 5  directed 
arcs  between  them.  The  directed  arcs  are  ordered  pairs  ( i,j )  where  i  and  j  are  nodes. 

The  length  of  the  directed  arc  (/,/)  is  denoted  as  ti; ,  for  example:  t24  =  6  .  In  this 

optimization  problem,  we  wish  to  find  the  shortest  path  through  the  network.  Let 
f.  =  the  minimum  travel  time  from  node  i  to  node  9 

By  definition,  f9=  0 ,  and  we  interpret 

tij  +  fj 

as  the  travel  time  of  the  path  from  node  i  to  node  9  that  first  traverses  arc  (/,/)  and  then 
travels  as  quickly  as  possible  from  node  j  to  node  9.  As  this  is  a  path  from  node  i  to  node 
9,  its  travel  time  must  be  at  least  as  large  as  f.  (otherwise  f.  is  not  a  minimum  path.)  and 

in  reality  t{j  +  /.  could  be  a  lot  longer  than  f. .  Thus 

f^tj+fj  for 

There  is  a  fastest  path  from  node  i  to  node  9  and  it  traverses  some  arc  (/,  j)  first  and  then 
gets  from  node  j  to  node  9  as  quickly  as  possible.  So  some  j  satisfies  the  above  inequality 
as  an  equality,  and  this  is  the  functional  equation: 

f  =  min {f+fj}  for  i* 9  (18) 

The  set  of  j  over  which  the  right  hand  side  of  equation  (18)  is  to  be  minimized  occurs 
over  those  j  for  which  (/',/)  is  an  arc  (kind  of  hard  to  minimize  otherwise).  Each  of  those 
nodes  has  a  corresponding  /  (the  min  path  from  j  to  the  end)  and  each  of  the  arcs  to 

node  j  has  a  corresponding  length  ttj  .  One  of  those  j  happens  to  be  the  minimum  path, 

and  that  path  determines  the  value  for  f..  To  solve  the  problem,  you  embed  it  in  the 

larger  problem  of  finding  the  minimum  path  from  every  node.  Starting  from  node  /  =  8, 
using  equation  (18)  for  every  node  to  backtrack  the  answer:  (1,3, 4, 5, 7, 9). 

Algorithm  Development 

Now  the  question  remains,  how  do  we  solve  this  problem  on  a  computer?  Well,  first  we 
must  put  the  directed  network  in  Figure  10  into  a  cost  matrix,  C,  for  calculation  purposes. 
We  set  up  the  cost  matrix  like  this: 


C(m,n ) 


1 

2 

NaN 

NaN 

NaN 

6 

12 

NaN 

3 

NaN 

4 

NaN 

4 

3 

15 

7 

NaN 

7 

1 NaN 

NaN 

NaN 

7 

15 

NaN 

NaN 

3 

NaN 

NaN 

10 

NaN 

NaN 

NaN 

There  are  eight  rows  in  the  matrix,  and  the  row  number  m  is  the  same  as  the  node 
number:  i=m.  We  only  need  eight  rows,  because  we  already  know  the  value  for  f9  =  0 . 

We  could  have  made  a  square  matrix,  so  the  column  number  would  be  the  same  as  j,  but 
this  would  have  been  a  really  sparse  matrix.  Looking  at  Figure  10,  we  see  that  the  nodes 
are  labeled  so  that  each  arc  (/,/)  has  i  <  j.  This  means  that  C(i,j)  would  be  empty  for  all 
j  <  i .  That  is  why  we  decided  instead  to  make  the  column  number  n  refer  to  j  =  i  +  n  . 
For  example,  if  we  are  in  row  2  (node  2)  the  third  column  is  the  directed  arc  (2,  2+3)  = 
(2,5)  which  has  length  t25  =12.  We  only  have  to  make  a  total  of  4  columns  because  no 

node  has  more  than  four  directed  arcs  emanating  from  it.  NaN  is  a  nonexistent  entry,  a 
placeholder  which  is  ignored  when  calculating  the  minimum.  Thus,  to  find  ttj  from 

C(m,n)  we  substitute: 

t(i,j)  =  C(m,m  +  n) 

For  computational  compatibility,  we  need  to  pad  f.  We  know  that  f9  =  0 ,  but  we  also 
need  /s+4  =  /8+3  =  /s+2  =  NaN .  This  keeps  the  nonexistent  paths  nonexistent 
because:  NaN  +  a  -  NaN .  At  last  we  find  the  total  cost  ttj  +  f  and  minimize  to  find  f. 

by  the  functional  equation  (18): 

for  m  =  8:-l:l 

for  n  =  1:4 

Total _Cost(m,n)  =C(m,n)  +f(m+n); 

end 

f(m)  =min(Total _Cost(m, :)); 

end 


We  have  now  built  a  Total  Cost  matrix  which  gives  the  cost  for  each  path  from  a  given 
node  i=m  to  the  end.  We  then  take  the  min  of  that  row  (the  equivalent  of  taking  the  min 
over  j  as  shown  in  the  functional  equation  (18))  to  find  the  minimum  path  from  a  given 
node  to  the  end.  When  we  are  done,  we  not  only  know  the  minimum  path  from  the 
beginning  node  to  the  end,  we  also  know  the  minimum  path  from  every  other  node  to  the 
end.  This  is  a  basic  property  of  dynamic  programming:  you  embed  the  smaller  problem 
of  finding  the  minimum  path  into  the  larger  problem  of  finding  the  minimum  path  from 
every  node,  and  you  then  solve  the  smaller  problem  by  first  solving  the  larger  problem. 
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